1. Technical Field of the Invention
The present invention relates to computer graphics, and more particular, to graphics texture processing methods, apparatus and computer program products.
2. Description of Related Art
The real-time rendering of three-dimensional graphics has a number of appealing applications on mobile terminals, including games, man-machine interfaces, messaging and m-commerce. Since three-dimensional rendering is a computationally expensive task, dedicated hardware must often be built to reach sufficient performance. Innovative ways of lowering the complexity and bandwidth usage of this hardware architecture are thus of great importance.
One standard way of increasing the level of realism in real-time rendering is to apply textures to the surfaces. This can be done in a variety of ways, including environment mapping, bump mapping, texturing via automatic texture coordinate generation, projection-based texturing, and more. Common to all different ways of using textures is that high quality texture filtering may be needed to reduce aliasing problems. Aliasing may be extremely disturbing for the human viewer, especially for dynamic scenes.
A commonly used scheme for texture filtering that is implemented in hardware is mipmapping and, in the form of trilinear mipmapping, this provides relatively high quality filtering. A disadvantage is that overblurring can occur. However, it should be noted that this may be preferred over aliasing. Mipmapping is discussed for example by Lance Williams in “Pyramidal Parametrics,” (Computer Graphic, SIGGRAPH '83 Proceedings, pp. 1-11, July 1983), the disclosure of which is hereby incorporated herein in its entirety by reference.
Trilinear mipmapping requires eight texels that are filtered into one color that is used, and thus, in the worst case, eight memory reads can be needed before the filtered color is obtained. Texture caches with prefetching can hide this problem, but such techniques may be expensive in hardware.
Anisotropic filtering schemes may reduce or eliminate overblurring by, e.g., filtering several trilinear mipmap samples. The ultimate goal is that those together may perfectly cover the quadrilateral that is obtained from projecting a pixel to texture space. While increasing quality, a number of memory reads may increase dramatically. Common numbers range from 16-128 texel reads.
Aspects of rendering, texture filtering, texture mapping, and/or parametrics are discussed in the following references: Akenine-Möller, Tomas, and Eric Haines, Real-Time Rendering, 2nd edition, June 2002; Beers, Andrew C., Maneesh Agrawala, and Navin Chaddha, “Rendering from Compressed Textures,” Computer Graphics (SIGGRAPH 96 Proceedings), pp. 373-378, August 1996; Behrens, Uwe, “Averaged Area Tables for Texture Filtering,” SIGGRAPH 2001 Conference Abstracts and Applications, p. 150, 2001; Crow, Franklin C., “Summed-Area Tables for Texture Mapping,” Computer Graphics (SIGGRAPH '84 Proceedings), pp. 207-212, July 1984; McCabe, Dan, and John Brothers, “DirectX 6 Texture Map Compression,” Game Developer Magazine, vol. 5, no. 8, pp. 42-46, August 1998; and Williams, Lance, “Pyramidal Parametrics,” Computer Graphics (SIGGRAPH '83 Proceedings), pp. 1-11, July 1983. The disclosures of each of these references are hereby incorporated herein in their entirety by reference.
For mobile platforms, memory accesses can be extremely expensive. On a standard computer graphics architecture, more than two thirds of the memory accesses can be due to texturing. In the example of trilinear mipmapping for instance, texturing may require eight memory accesses, whereas the rest of the rasterizer may require another three (z-read, z-write and colorbuffer write). With eleven memory accesses per pixel, the memory bandwidth need be very high to obtain reasonable performance. A high clock speed may cost a lot of power and may not be suitable for mobile platforms. It may thus be desirable to reduce the number of memory accesses that are needed for texturing, while maintaining a texture quality that is similar to trilinear mipmapping.
Texture compression is a means often used to save bandwidth in graphics hardware. The idea is that the texel values (which may comprise of, for example, RGB color values, gray scale values or other graphics representations) are stored in a compressed format in the memory, and this can reduce the bandwidth needed when the texture is sent over the bus. When the data reaches the destination, a hardware mechanism can decompress the texel data.
Pioneering work is discussed by Andrew C. Beers et al. in “Rendering from Compressed Textures,” (Computer Graphics, SIGGRAPH 96 Proceedings, pp. 373-378, August 1996), the disclosure of which is hereby incorporated herein in its entirety by reference. Beers et al. discuss using vector quantization, and training the vector set using a number of images.
S3 Graphics Co., Ltd., has invented a texture compression scheme, which is often called S3TC (S3 Texture Compression). S3TC is discussed for example by McGabe et al. in “DirectX 6 Texture Map Compression” (Game Developer Magazine, vol. 5, no. 8, pp. 42-46, August 1998), the disclosure of which is hereby incorporated herein in its entirety by reference. The S3TC has been incorporated into DirectX, where it is called DXTC. The S3TC scheme compresses textures, that is, images that are “glued” onto geometric primitives (e.g., triangles) during rendering. The texture image is divided into 4×4 texel blocks, and each such block is compressed individually into 8 bytes. This may be done as follows. First, two texel reference values are stored in 16 bits each. Each texel reference value may for example be color encoded as red (5 bits), green (6 bits), and blue (5 bits). During decompression, an additional two texel reference values are computed in between the two stored texel reference values. This means that each 4×4 block has access to a total of four texel reference values (e.g., colors). Therefore, each of the 16 (4×4) texels can be encoded as an index into this “palette” of 4 texel reference values. Each index can be coded with 2 bits, and we call these indices texel index values. Each block requires 16+16+4*4*2=64 bits=8 bytes. Without compression, each block requires 3 bytes per texel, which gives 3*16=48 bytes. The compression ratio is thus 1:6.
S3TC decompression may proceed as follows, assuming rendering to a 24-bit display. Colors stored with 16 bits may be converted to 24 bits by zero-padding, although other ways to do this exist.
Assuming that the decompression algorithm receives a 64-bit sequence (00000 000000 00000 10000 100000 00000 00 00 00 00 01 01 01 01 10 10 10 10 11 11 11 11) as an input, the first two 16-bit values are the two texel reference values (colors), which, after zero padding, equals RGB1=(0, 0, 0) and RGB2=(128, 128, 0). Two more reference values may be found by taking:RGB3=⅔*RGB1+⅓*RGB2=(45 45 0) andRGB4=⅓*RGB1+⅔*RGB2=(82 82 0).
Next come the texel index values, one for each texel, each coded with two bits, in row order. In this case, all the texels in the first row have the value 00, which means that RGB1=(0, 0, 0) will be picked. The next row is all RGB2 etc. Finally, the decoded block will have the texel values shown in Table 0:
TABLE 0(0, 0, 0)(0, 0, 0)(0, 0, 0)(0, 0, 0)(128, 128, 0)(128, 128, 0)(128, 128, 0)(128, 128, 0)(45, 45, 0)(45, 45, 0)(45, 45, 0)(45, 45, 0)(82, 82, 0)(82, 82, 0)(82, 82, 0)(82, 82, 0)